Assessment of Evaluation Methods for Binary Classification Modeling

In this paper, we examine three common modeling techniques – Linear Discriminant Analysis, Logistic Regression Analysis and Neural Networks. Although a wellestablished literature exists examining the relative performances of these techniques, limited organized research attention has been given to the evaluation methods used to determine model and technique superiority. We compare and contrast models developed using these techniques, specifically examining their respective predictive classification accuracy through three methods of evaluation – Classification Rates, The KolmorgorovSmirnov Test and ROC curves. As our results revealed, the selection of a ‘best’ model is contingent on the chosen evaluation method. Consequently, analysts would be well served to develop an understanding of the circumstances under which each evaluation method should be utilized. Subject Areas: Classification Models, Model Evaluation and Credit Modeling